Threat disposition analysis and modeling using supervised machine learning

ABSTRACT

An enhanced threat disposition analysis technique is provided. In response to receipt of a security threat, a threat disposition score (TDS) is retrieved. The threat disposition score is generated from a machine learning scoring model that is built from information about historical security threats, including historical disposition of one or more alerts associated with the historical security threats. The system augments an alert to include the threat disposition score, optionally together with a confidence level, to generate an enriched alert. The enriched alert is then presented to the security analyst for handling directly. Depending on the TDS (and its confidence level), the analyst may be able to respond to the threat immediately, i.e., without further detailed investigation. Preferably, the machine learning model is updated continuously as the system handles security threats, thereby increasing the predictive benefit of the TDS scoring.

BACKGROUND Technical Field

This disclosure relates generally to cybersecurity offense analytics.

Background of the Related Art

Enterprise security is a complex problem requiring the coordinationacross security policies, controls, threat models and threat detectionscenarios (use cases). The implementation of these policies, models andcontrols requires extensive use of threat monitoring technologies andsecurity devices, as well as human resources that have security,business and technical skills. In particular, the ever increasing numberof threats at scale requires automation in support of security analysts,who are responsible for preventing, detecting and responding to thesethreats. In most cases, the analyst must manually search through a widerange of data sources (some private, many public), review past threatevents and how they were handled, check for duplicate events, currentlyopen similar events and a knowledge database, etc., to determine anappropriate response procedure to handle this information. This processof data collection, analysis, and determining the final disposition ofthe alert, is time consuming and tedious for an analyst.

There are a variety of tools that exist for threat monitoring to analyzea wide range of data sources (including structured data, unstructureddata, semi-structured data, and reference data) to identify patternsthat are indicative of threats, security policy and control anomalies.When these threats and/or anomalies are detected, actionable alerts arecreated. In many cases, the number of alerts exceeds the capacity of thesecurity analyst to effectively and efficiently handle them. SecurityOperations Center (SOC) analysts are responsible for this process, andthis is typically done by taking a closer look at the raw dataassociated with the alert, including a review of both contextual data,as well as the raw data from the data sources that triggered the alert.As noted, this data collection and investigation is time-consuming, andit often requires complex correlation analysis. This fact correlationcan include information that is general to the threat or anomaly, but itcan also require very specific information about a customer, theirassets, and any other special instructions the customer may haveprovided regarding the proper alert handling. Additionally, the securityanalyst may often need to do additional research to understand thenature of the threat, the vector of the attack, and so forth, to discernwhether the target is truly vulnerable.

Once all known factors are considered, the security analyst must thendetermine the optimal disposition for a specific alert. There are arange of possible dispositions including, but not limited to identifyingthe alert as one of: a duplicate, a false positive, a currently opencase, a new case (first of a kind), and a known alert. For each of thesedispositions, there are also a range of actions that the analyst canrecommend including, for example: closing the alert with no furtheraction, holding the alert for further investigation, and escalating thealert for additional review. In each of these cases, the analyst alsomay be able to recommend the specific mitigation and remediationactivities that are needed to deal with the alert.

It is known in the prior art to provide systems that can classify theseverity of an alert, but typically these systems use static algorithmsthat simply calculate a score based on pre-defined attributes built intoan alert rule. Moreover, these systems only provide pre-definedrecommendations on the handling of the alert, and they do not includethe ability to learn about the likely disposition of the alert.

BRIEF SUMMARY

The subject matter herein provides a mechanism and method to reduce thetime required for security analyst alert investigation, preferably byenriching threat data with additional contextual information, with aprimary goal being reducing alert disposition error rates. To this end,machine learning (ML) is used to augment a security threat monitoringplatform. Preferably, the machine learning is trained usingpreviously-handled alerts and, in particular, by analyzing historicaldisposition of these alerts. Preferably, these analytics supplementalert information to generate a data-driven threat disposition score(TDS) that helps the analyst characterize the alert he or she isanalyzing, e.g., to determine the likelihood that is a false positiveversus a potential security incident. As the machine learning continues,the accuracy of the TDS continues to increase as the system learns fromSOC analyst actions (escalation vs. closing as false positive), as wellas feedback on alert handling (e.g., from higher level security analystson the actions taken by the front line security analyst. Over time, andas the machine learning adapts, the algorithms improve their accuracyand predict alert dispositions with high accuracy levels.

According to a first aspect of this disclosure, a method for threatdisposition analysis is provided. The method begins in response toreceipt of a security threat. In particular, upon receipt, a threatdisposition score (TDS) is retrieved. The threat disposition score isgenerated from a machine learning scoring model that is built frominformation about historical security threats, including historicaldisposition of one or more alerts associated with the historicalsecurity threats. The system then augments an alert to include thethreat disposition score, optionally together with a confidence level,to generate an enriched alert. The enriched alert is then presented tothe security analyst for handling directly. Depending on the TDS (andits confidence level), the analyst may be able to respond to the threatimmediately, i.e., without further detailed investigation. Preferably,the machine learning model is updated continuously as the system handlessecurity threats, thereby increasing the predictive benefit of the TDSscoring.

According to a second aspect of this disclosure, an apparatus forprocessing security event data is described. The apparatus comprises aset of one or more hardware processors, and computer memory holdingcomputer program instructions executed by the hardware processors toperform a set of operations such as described above.

According to a third aspect of this disclosure, a computer programproduct in a non-transitory computer readable medium for use in a dataprocessing system for processing security event data is described. Thecomputer program product holds computer program instructions executed inthe data processing system and operative to perform operations such asdescribed above.

The foregoing has outlined some of the more pertinent features of thesubject matter. These features should be construed to be merelyillustrative. Many other beneficial results can be attained by applyingthe disclosed subject matter in a different manner or by modifying thesubject matter as will be described.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the subject matter and theadvantages thereof, reference is now made to the following descriptionstaken in conjunction with the accompanying drawings, in which:

FIG. 1 depicts an exemplary block diagram of a distributed dataprocessing environment in which exemplary aspects of the illustrativeembodiments may be implemented;

FIG. 2 is an exemplary block diagram of a data processing system inwhich exemplary aspects of the illustrative embodiments may beimplemented;

FIG. 3 illustrates a security intelligence platform in which thetechniques of this disclosure may be practiced;

FIG. 4 depicts a Level 1 security threat monitoring operation in a datacenter operating environment according to known techniques;

FIG. 5 depicts the technique of this disclosure wherein supervisedmachine learning is used to augment the security threat monitoringsystem in FIG. 4 ;

FIG. 6 is a high level process flow depicting how to create a scoringmodel according to this disclosure;

FIG. 7 is a high level process flow depicting how to use the trainedmodel to facilitate alert disposition by the security analyst;

FIG. 8 is a high level process flow depicting how to update the scoringmodel;

FIG. 9 is a high level process flow describing how to configureattributes for the TDS scoring model according to this disclosure; and

FIG. 10 is a representative portion of a display screen showing an alertprovided to an analyst using the technique of this disclosure.

DETAILED DESCRIPTION OF AN ILLUSTRATIVE EMBODIMENT

With reference now to the drawings and in particular with reference toFIGS. 1-2 , exemplary diagrams of data processing environments areprovided in which illustrative embodiments of the disclosure may beimplemented. It should be appreciated that FIGS. 1-2 are only exemplaryand are not intended to assert or imply any limitation with regard tothe environments in which aspects or embodiments of the disclosedsubject matter may be implemented. Many modifications to the depictedenvironments may be made without departing from the spirit and scope ofthe present invention.

With reference now to the drawings, FIG. 1 depicts a pictorialrepresentation of an exemplary distributed data processing system inwhich aspects of the illustrative embodiments may be implemented.Distributed data processing system 100 may include a network ofcomputers in which aspects of the illustrative embodiments may beimplemented. The distributed data processing system 100 contains atleast one network 102, which is the medium used to provide communicationlinks between various devices and computers connected together withindistributed data processing system 100. The network 102 may includeconnections, such as wire, wireless communication links, or fiber opticcables.

In the depicted example, server 104 and server 106 are connected tonetwork 102 along with storage unit 108. In addition, clients 110, 112,and 114 are also connected to network 102. These clients 110, 112, and114 may be, for example, personal computers, network computers, or thelike. In the depicted example, server 104 provides data, such as bootfiles, operating system images, and applications to the clients 110,112, and 114. Clients 110, 112, and 114 are clients to server 104 in thedepicted example. Distributed data processing system 100 may includeadditional servers, clients, and other devices not shown.

In the depicted example, distributed data processing system 100 is theInternet with network 102 representing a worldwide collection ofnetworks and gateways that use the Transmission ControlProtocol/Internet Protocol (TCP/IP) suite of protocols to communicatewith one another. At the heart of the Internet is a backbone ofhigh-speed data communication lines between major nodes or hostcomputers, consisting of thousands of commercial, governmental,educational and other computer systems that route data and messages. Ofcourse, the distributed data processing system 100 may also beimplemented to include a number of different types of networks, such asfor example, an intranet, a local area network (LAN), a wide areanetwork (WAN), or the like. As stated above, FIG. 1 is intended as anexample, not as an architectural limitation for different embodiments ofthe disclosed subject matter, and therefore, the particular elementsshown in FIG. 1 should not be considered limiting with regard to theenvironments in which the illustrative embodiments of the presentinvention may be implemented.

With reference now to FIG. 2 , a block diagram of an exemplary dataprocessing system is shown in which aspects of the illustrativeembodiments may be implemented. Data processing system 200 is an exampleof a computer, such as client 110 in FIG. 1 , in which computer usablecode or instructions implementing the processes for illustrativeembodiments of the disclosure may be located.

With reference now to FIG. 2 , a block diagram of a data processingsystem is shown in which illustrative embodiments may be implemented.Data processing system 200 is an example of a computer, such as server104 or client 110 in FIG. 1 , in which computer-usable program code orinstructions implementing the processes may be located for theillustrative embodiments. In this illustrative example, data processingsystem 200 includes communications fabric 202, which providescommunications between processor unit 204, memory 206, persistentstorage 208, communications unit 210, input/output (I/O) unit 212, anddisplay 214.

Processor unit 204 serves to execute instructions for software that maybe loaded into memory 206. Processor unit 204 may be a set of one ormore processors or may be a multi-processor core, depending on theparticular implementation. Further, processor unit 204 may beimplemented using one or more heterogeneous processor systems in which amain processor is present with secondary processors on a single chip. Asanother illustrative example, processor unit 204 may be a symmetricmulti-processor (SMP) system containing multiple processors of the sametype.

Memory 206 and persistent storage 208 are examples of storage devices. Astorage device is any piece of hardware that is capable of storinginformation either on a temporary basis and/or a permanent basis. Memory206, in these examples, may be, for example, a random access memory orany other suitable volatile or non-volatile storage device. Persistentstorage 208 may take various forms depending on the particularimplementation. For example, persistent storage 208 may contain one ormore components or devices. For example, persistent storage 208 may be ahard drive, a flash memory, a rewritable optical disk, a rewritablemagnetic tape, or some combination of the above. The media used bypersistent storage 208 also may be removable. For example, a removablehard drive may be used for persistent storage 208.

Communications unit 210, in these examples, provides for communicationswith other data processing systems or devices. In these examples,communications unit 210 is a network interface card. Communications unit210 may provide communications through the use of either or bothphysical and wireless communications links.

Input/output unit 212 allows for input and output of data with otherdevices that may be connected to data processing system 200. Forexample, input/output unit 212 may provide a connection for user inputthrough a keyboard and mouse. Further, input/output unit 212 may sendoutput to a printer. Display 214 provides a mechanism to displayinformation to a user.

Instructions for the operating system and applications or programs arelocated on persistent storage 208. These instructions may be loaded intomemory 206 for execution by processor unit 204. The processes of thedifferent embodiments may be performed by processor unit 204 usingcomputer implemented instructions, which may be located in a memory,such as memory 206. These instructions are referred to as program code,computer-usable program code, or computer-readable program code that maybe read and executed by a processor in processor unit 204. The programcode in the different embodiments may be embodied on different physicalor tangible computer-readable media, such as memory 206 or persistentstorage 208.

Program code 216 is located in a functional form on computer-readablemedia 218 that is selectively removable and may be loaded onto ortransferred to data processing system 200 for execution by processorunit 204. Program code 216 and computer-readable media 218 form computerprogram product 220 in these examples. In one example, computer-readablemedia 218 may be in a tangible form, such as, for example, an optical ormagnetic disc that is inserted or placed into a drive or other devicethat is part of persistent storage 208 for transfer onto a storagedevice, such as a hard drive that is part of persistent storage 208. Ina tangible form, computer-readable media 218 also may take the form of apersistent storage, such as a hard drive, a thumb drive, or a flashmemory that is connected to data processing system 200. The tangibleform of computer-readable media 218 is also referred to ascomputer-recordable storage media. In some instances,computer-recordable media 218 may not be removable.

Alternatively, program code 216 may be transferred to data processingsystem 200 from computer-readable media 218 through a communicationslink to communications unit 210 and/or through a connection toinput/output unit 212. The communications link and/or the connection maybe physical or wireless in the illustrative examples. Thecomputer-readable media also may take the form of non-tangible media,such as communications links or wireless transmissions containing theprogram code. The different components illustrated for data processingsystem 200 are not meant to provide architectural limitations to themanner in which different embodiments may be implemented. The differentillustrative embodiments may be implemented in a data processing systemincluding components in addition to or in place of those illustrated fordata processing system 200. Other components shown in FIG. 2 can bevaried from the illustrative examples shown. As one example, a storagedevice in data processing system 200 is any hardware apparatus that maystore data. Memory 206, persistent storage 208, and computer-readablemedia 218 are examples of storage devices in a tangible form.

In another example, a bus system may be used to implement communicationsfabric 202 and may be comprised of one or more buses, such as a systembus or an input/output bus. Of course, the bus system may be implementedusing any suitable type of architecture that provides for a transfer ofdata between different components or devices attached to the bus system.Additionally, a communications unit may include one or more devices usedto transmit and receive data, such as a modem or a network adapter.Further, a memory may be, for example, memory 206 or a cache such asfound in an interface and memory controller hub that may be present incommunications fabric 202.

Computer program code for carrying out operations of the presentinvention may be written in any combination of one or more programminglanguages, including an object-oriented programming language such asJava™, Smalltalk, C++ or the like, and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer, or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Those of ordinary skill in the art will appreciate that the hardware inFIGS. 1-2 may vary depending on the implementation. Other internalhardware or peripheral devices, such as flash memory, equivalentnon-volatile memory, or optical disk drives and the like, may be used inaddition to or in place of the hardware depicted in FIGS. 1-2 . Also,the processes of the illustrative embodiments may be applied to amultiprocessor data processing system, other than the SMP systemmentioned previously, without departing from the spirit and scope of thedisclosed subject matter.

As will be seen, the techniques described herein may operate inconjunction within the standard client-server paradigm such asillustrated in FIG. 1 in which client machines communicate with anInternet-accessible Web-based portal executing on a set of one or moremachines. End users operate Internet-connectable devices (e.g., desktopcomputers, notebook computers, Internet-enabled mobile devices, or thelike) that are capable of accessing and interacting with the portal.Typically, each client or server machine is a data processing systemsuch as illustrated in FIG. 2 comprising hardware and software, andthese entities communicate with one another over a network, such as theInternet, an intranet, an extranet, a private network, or any othercommunications medium or link. A data processing system typicallyincludes one or more processors, an operating system, one or moreapplications, and one or more utilities. The applications on the dataprocessing system provide native support for Web services including,without limitation, support for HTTP, SOAP, XML, WSDL, UDDI, and WSFL,among others. Information regarding SOAP, WSDL, UDDI and WSFL isavailable from the World Wide Web Consortium (W3C), which is responsiblefor developing and maintaining these standards; further informationregarding HTTP and XML is available from Internet Engineering Task Force(IETF). Familiarity with these standards is presumed.

Security Intelligence Platform with Incident Forensics

A known type of security intelligence platform is illustrated in FIG. 3. Generally, the platform provides search-driven data exploration,session reconstruction, and forensics intelligence to assist securityincident investigations. In pertinent part, the platform 300 comprises aset of packet capture appliances 302, an incident forensics moduleappliance 304, a distributed database 306, and a security intelligenceconsole 308. The packet capture and module appliances are configured asnetwork appliances, or they may be configured as virtual appliances. Thepacket capture appliances 302 are operative to capture packets off thenetwork (using known packet capture (pcap) application programminginterfaces (APIs) or other known techniques), and to provide such data(e.g., real-time log event and network flow) to the distributed database306, where the data is stored and available for analysis by theforensics module 304 and the security intelligence console 308. A packetcapture appliance operates in a session-oriented manner, capturing allpackets in a flow, and indexing metadata and payloads to enable fastsearch-driven data exploration. The database 306 provides a forensicsrepository, which distributed and heterogeneous data sets comprising theinformation collected by the packet capture appliances. The console 308provides a web- or cloud-accessible user interface (UI) that exposes a“Forensics” dashboard tab to facilitate an incident investigationworkflow by an investigator. Using the dashboard, an investigatorselects a security incident. The incident forensics module 304 retrievesall the packets (including metadata, payloads, etc.) for a selectedsecurity incident and reconstructs the session for analysis.

A representative commercial product that implements an incidentinvestigation workflow of this type is IBM® Security QRadar® IncidentForensics V7.2.3 (or higher). Using this platform, an investigatorsearches across the distributed and heterogeneous data sets stored inthe database, and receives a unified search results list. The searchresults may be merged in a grid, and they can be visualized in a“digital impression” tool so that the user can explore relationshipsbetween identities.

In particular, a typical incident forensics investigation to extractrelevant data from network traffic and documents in the forensicrepository is now described. According to this approach, the platformenables a simple, high-level approach of searching and bookmarking manyrecords at first, and then enables the investigator to focus on thebookmarked records to identify a final set of records. In a typicalworkflow, an investigator determines which material is relevant. He orshe then uses that material to prove a hypothesis or “case” to developnew leads that can be followed up by using other methods in an existingcase. Typically, the investigator focuses his or her investigationthrough course-grained actions at first, and then proceeds to fine-tunethose findings into a relevant final result set. The bottom portion ofFIG. 3 illustrates this basic workflow. Visualization and analysis toolsin the platform may then be used to manually and automatically assessthe results for relevance. The relevant records can be printed,exported, or submitted processing.

As noted above, the platform console provides a user interface tofacilitate this workflow. Thus, for example, the platform provides asearch results page as a default page on an interface display tab.Investigators use the search results to search for and access documents.The investigator can use other tools to further the investigation. Oneof these tools is a digital impression tool. A digital impression is acompiled set of associations and relationships that identify an identitytrail. Digital impressions reconstruct network relationships to helpreveal the identity of an attacking entity, how it communicates, andwhat it communicates with. Known entities or persons that are found inthe network traffic and documents are automatically tagged. Theforensics incident module 304 is operative to correlate taggedidentifiers that interacted with each other to produce a digitalimpression. The collection relationships in a digital impression reportrepresent a continuously-collected electronic presence that isassociated with an attacker, or a network-related entity, or any digitalimpression metadata term. Using the tool, investigators can click anytagged digital impression identifier that is associated with a document.The resulting digital impression report is then listed in tabular formatand is organized by identifier type.

Generalizing, a digital impression reconstructs network relationships tohelp the investigator identify an attacking entity and other entitiesthat it communicates with. A security intelligence platform includes aforensics incident module that is operative to correlate taggedidentifiers that interacted with each other to produce a digitalimpression. The collection relationships in a digital impression reportrepresent a continuously-collected electronic presence that isassociated with an attacker, or a network-related entity, or any digitalimpression metadata term. Using the tool, investigators can click anytagged digital impression identifier that is associated with a document.The resulting digital impression report is then listed in tabular formatand is organized by identifier type.

Typically, an appliance for use in the above-described system isimplemented is implemented as a network-connected, non-display device.For example, appliances built purposely for performing traditionalmiddleware service oriented architecture (SOA) functions are prevalentacross certain computer environments. SOA middleware appliances maysimplify, help secure or accelerate XML and Web services deploymentswhile extending an existing SOA infrastructure across an enterprise. Theutilization of middleware-purposed hardware and a lightweight middlewarestack can address the performance burden experienced by conventionalsoftware solutions. In addition, the appliance form-factor provides asecure, consumable packaging for implementing middleware SOA functions.One particular advantage that these types of devices provide is tooffload processing from back-end systems. A network appliance of thistype typically is a rack-mounted device. The device includes physicalsecurity that enables the appliance to serve as a secure vault forsensitive information. Typically, the appliance is manufactured,pre-loaded with software, and then deployed within or in associationwith an enterprise or other network operating environment;alternatively, the box may be positioned locally and then provisionedwith standard or customized middleware virtual images that can besecurely deployed and managed, e.g., within a private or an on premisecloud computing environment. The appliance may include hardware andfirmware cryptographic support, possibly to encrypt data on hard disk.No users, including administrative users, can access any data onphysical disk. In particular, preferably the operating system (e.g.,Linux) locks down the root account and does not provide a command shell,and the user does not have file system access. Typically, the appliancedoes not include a display device, a CD or other optical drive, or anyUSB, Firewire or other ports to enable devices to be connected thereto.It is designed to be a sealed and secure environment with limitedaccessibility and then only be authenticated and authorized individuals.

An appliance of this type can facilitate Security Information EventManagement (SIEM). For example, IBM® Security QRadar® SIEM is anenterprise solution that includes packet data capture appliances thatmay be configured as appliances of this type. Such a device isoperative, for example, to capture real-time Layer 4 network flow datafrom which Layer 7 application payloads may then be analyzed, e.g.,using deep packet inspection and other technologies. It providessituational awareness and compliance support using a combination offlow-based network knowledge, security event correlation, andasset-based vulnerability assessment. In a basic QRadar STEMinstallation, the system such as shown in FIG. 3 is configured tocollect event and flow data, and generate reports. As noted, a user(e.g., an SOC analyst) can investigate offenses to determine the rootcause of a network issue.

Generalizing, Security Information and Event Management (SIEM) toolsprovide a range of services for analyzing, managing, monitoring, andreporting on IT security events and vulnerabilities. Such servicestypically include collection of events regarding monitored accesses andunexpected occurrences across the data network, and analyzing them in acorrelative context to determine their contribution to profiledhigher-order security events. They may also include analysis of firewallconfigurations, network topology and connection visualization tools forviewing current and potential network traffic patterns, correlation ofasset vulnerabilities with network configuration and traffic to identifyactive attack paths and high-risk assets, and support of policycompliance monitoring of network traffic, topology and vulnerabilityexposures. Some SIEM tools have the ability to build up a topology ofmanaged network devices such as routers, firewalls, and switches basedon a transformational analysis of device configurations processedthrough a common network information model. The result is a locationalorganization which can be used for simulations of security threats,operational analyses of firewall filters, and other applications. Theprimary device criteria, however, are entirely network- andnetwork-configuration based. While there are a number of ways to launcha discovery capability for managed assets/systems, and while containmentin the user interface is semi-automatically managed (that is, anapproach through the user interface that allows for semi-automated,human-input-based placements with the topology, and its display andformatting, being data-driven based upon the discovery of both initialconfigurations and changes/deletions in the underlying network), nothingis provided in terms of placement analytics that produce fully-automatedplacement analyses and suggestions.

FIG. 4 depicts a Security Operation Center (SOC) that provides Level 1security threat monitoring using an analytics platform 400 such as IBMQRadar. The platform 400 receives alerts (at step (1)) from a variety oflog sources 402, such as firewalls, intrusion detection and preventionsystems, antivirus systems, web proxies, and other systems and networkdevices. At step (2), the alerts are stored in an alert database 404. Atstep (3), the alerts are provided to a threat monitoring console 406that is manned by a security analyst 408. As is well-known, a SOCtypically is manned by different levels of security analysts. A Level 1(L1) analyst 408 is responsible for monitoring reported security events,and for closing or escalating those events according to SOC rules,policies and procedures. The security analyst 408 typically interactswith a client 410, which is the enterprise entity having an applicationthat is being monitored for security threats. Although not shown,typically the SOC has one or more additional levels of securityanalysts, such Level 2 (L2) and Level 3 (L3) analysts. Typically, L2security analysts handle escalations from L1 analysts and perform otheradministration and management functions, such as monitoring theperformance of the L1 analysts to ensure that security events arehandled timely, mentoring, and the like. Level 3 analysts handle furtherescalations (from L2 analysts), and provide additional higher-leveladministration and management functions in the SOC. Of course, thenumber of levels and the various tasks associated with each level may bevaried and implementation-specific.

As depicted, the L1 analyst makes a finding regarding an alert,typically with a goal of making this finding within about 15-20 minutesafter receiving the alert. Typically, the finding closes the alert (step5(a)) as a false positive, or escalation the alert (step 5(b)) as apossible attack. The false positive finding is stored in the alertdatabase 404. The attack finding typically is reported to the client 410whose application is affected. Depending on the implementation (e.g.,the SOC policy, the client procedure, etc.), some remediation or otheraction (step 6(b)) is taken; alternatively, the client 410 may indicatethat indeed the alert is a false positive and thus should be closed(step 6(c)). The responsive action 412 may be carried out in anautomated manner (e.g., programmatically), manually, or by a combinationof automation and manual operations. The action may be carried out bySOC personnel, by the client, or by a combination of SOC personnel andthe client. As also depicted, information regarding the response to thealert is also provided to a ticketing system 414, and such informationmay then be reported back to the security analyst (step 7(c)). Thesecurity analyst may then update the alert database (at step 8(c)) withthe information about how the alert was handled (or otherwise closed).Thus, the alert and its associated handling information is stored in thealert database 404 and available as a data source going forward.

Threat Disposition Analysis and Modeling Using Supervised MachineLearning

With the above as background, the following describes a threatdisposition analysis and modeling technique according to thisdisclosure. As will be seen, by using supervised machine learning asdescribed herein, the time required for threat investigation prior toremediation response is significantly reduced.

The preferred approach is depicted in FIG. 5 , which depict how the L1security threat monitoring technique in FIG. 4 is augmented withsupervised machine learning according to this disclosure. FIG. 5 depictsa Security Operation Center (SOC) that provides Level 1 security threatmonitoring using an analytics platform 500. Once again, the platform 500receives alerts (at step (1)) from one or more log sources 502. At step(2), the alerts are stored to the alert database 504. At step (3), thealerts are provided to the threat monitoring console 506 that is mannedby the L1 security analyst 510. The security analyst 508 interacts withthe client 510, as previously described. In particular, and as depicted,the L1 analyst makes a finding regarding an alert, and a goal of thetechnique of this disclosure is to reduce significantly the time neededfor the analyst to make his or her initial finding. As before, typicallythe finding closes the alert (step 5(a)) as a false positive, orescalates the alert (step 5(b)) as a possible attack. The false positivefinding is stored in the alert database 504. The attack findingtypically also is reported to the client 510 whose application isaffected. Once again, and depending on the implementation (e.g., the SOCpolicy, the client procedure, etc.), some remediation or other action(step 6(b)) is taken; alternatively, the client 510 indicates that thealert is a false positive and thus should be closed (step 6(c)). Theresponsive action 512 is carried out, the information regarding theresponse to the alert provided to the ticketing system 514, and theinformation is reported back to the security analyst (step 7(c)), all aspreviously described. The security analyst updates the alert database(at step 8(c)) with the information about how the alert was handled (orotherwise closed).

Unlike the technique shown in FIG. 4 , the approach of this disclosureuses machine learning techniques to enhance the threat dispositionanalysis. Machine learning (ML) algorithms iteratively learn from data,thus allowing computers to find hidden insights without being explicitlyprogrammed where to look. Machine Learning is essentially teaching thecomputer to solve problems by creating algorithms that learn by lookingat hundreds or thousands of examples, and then using that experience tosolve the same problem in new situations. Machine Learning tasks aretypically classified into the following three broad categories,depending on the nature of the learning signal or feedback available toa learning system: supervised learning, unsupervised learning, andreinforcement learning. In supervised learning, the algorithm trains onlabeled historic data and learns general rules that map input tooutput/target. In particular, the discovery of relationships between theinput variables and the label/target variable in supervised learning isdone with a training set. The computer/machine learns from the trainingdata. In this approach, a test set is used to evaluate whether thediscovered relationships hold and the strength and utility of thepredictive relationship is assessed by feeding the model with the inputvariables of the test data and comparing the label predicted by themodel with the actual label of the data. The most widely used supervisedlearning algorithms are Support Vector Machines, Linear Regression,Logistic Regression, Naive Bayes, and Neural Networks.

In unsupervised machine learning, the algorithm trains on unlabeleddata. The goal of these algorithms is to explore the data and find somestructure within. The most widely used unsupervised learning algorithmsare Cluster Analysis and Market Basket Analysis. In reinforcementlearning, the algorithm learns through a feedback system. The algorithmtakes actions and receives feedback about the appropriateness of itsactions and based on the feedback, modifies the strategy and takesfurther actions that would maximize the expected reward over a givenamount of time.

The following provides additional details regarding supervised machinelearning, which is the preferred technique used in the threatdisposition analysis approach of this disclosure. As noted above,supervised learning is the machine learning task of inferring a functionfrom labeled training data. The training data consist of a set oftraining examples. In supervised learning, typically each example is apair consisting of an input object (typically a vector), and a desiredoutput value (also called the supervisory signal). A supervised learningalgorithm analyzes the training data and produces an inferred function,which can be used for mapping new examples. An optimal scenario allowsfor the algorithm to correctly determine the class labels for unseeninstances. This requires the learning algorithm to generalize reasonablyfrom the training data to unseen situations.

To solve problem of supervised learning, one has to perform thefollowing steps: Determine the type of training examples. Before doinganything else, the user should decide what kind of data is to be used asa training set. Gather a training set; the training set needs to berepresentative of the real-world use of the function. Thus, a set ofinput objects is gathered and corresponding outputs are also gathered,either from human experts or from measurements. The, determine the inputfeature representation of the learned function. The accuracy of thelearned function depends strongly on how the input object isrepresented. Typically, the input object is transformed into a featurevector, which contains a number of features that are descriptive of theobject. The structure of the learned function and corresponding learningalgorithm are then determined. For example, the engineer may choose touse support vector machines or decision trees. The learning algorithm isthen run on the gathered training set. Some supervised learningalgorithms require the user to determine certain control parameters.These parameters may be adjusted by optimizing performance on a subset(called a validation set) of the training set, or via cross-validation.The accuracy of the learned function is then evaluated. After parameteradjustment and learning, the performance of the resulting function ismeasured on a test set that is separate from the training set.

Referring now back to FIG. 5 , in this approach herein informationcollected in the alert database 504 is provided to a machinelearning/training sub-system 516, which uses the information aboutalerts and prior alert handling to build a prediction model 518 that isthen provided to the security analysts (e.g., via the threat monitoringconsole 506) to reduce the time needed for the security analyst toidentify, categorize, prioritize and investigate events for theclient(s).

In a preferred approach, and for each threat detected (e.g. by a SIEM,an enterprise security Tool, any other Big Data tool) and presented tothe SOC analyst in the threat monitoring console 506 as an alert, thedata associated with this alert is enriched using the historicalinformation on how this alert has been handled previously. Thisenrichment is provided by the machine learning/training sub-system 516,which as noted above outputs the prediction model 518. Information fromthe model is summarized for the SOC analyst, typically in the form of areified value, referred to herein as a threat disposition score (TDS).Preferably, the TDS is enabled by a set of one or more supervisedmachine learning (ML) algorithms. Without limitation, preferably the MLalgorithm(s) create the prediction model 518 by taking into account dataabout historical alerts, e.g., what action the SOC analyst took on analert (e.g., escalation, closing, holding for further analysis, etc.),any feedback on alert handling (e.g., from L2 or L3 analysts based onthe L1 analyst action), as well as a variety of attributes regarding thenature of the alert itself. The system then continuously learns (e.g.,from new inputs) to improve and update its training model 518 on aregular basis. As the richness of historical data grows, the MLalgorithms in the machine learning/training sub-system 516 themselvesevolve to become more accurate at scoring new threats. Preferably, thisfeedback loop is enhanced further by evaluating an effectiveness of acalculated TDS in comparison to a remediation action taken by the SOCanalyst and vetted by feedback on alert handling (e.g., from L2 or L3analysts). Thus, for example, a scenario in which a TDS was at odds withthe remediation action taken allows the system to adjust and improve thetraining model 518, thereby improving performance (by further reducingthe response time). As another example, when a higher level analystresponds to an escalated alert and determines a correct alertdisposition (e.g. a L2 or L3 analyst affirms the alert is an actualthreat or requests to close the alert even though it was escalated (i.e.false positive)), this valuable feedback is provided to the machinelearning and reflected in an updated prediction model, thereby furtherimproving the accuracy of the predicted alert disposition as indicatedby the TDS.

The prediction model is sometimes referred to herein as a scoring model.FIG. 6 depicts a technique for creating the scoring model according toan embodiment. Here, the alert database 600 is configured to export datato a train scoring model function 602, which comprises one or moremachine learning algorithms. As depicted at 606, typically the dataexported from the database 600 includes a set of attributes required inthe model, as well as an initial training data set. Training (operation602) involves running the machine learning algorithm(s) with theprovided training set to generate an initial version of the model 604.The training is depicted at 608, and additional data from data sourcesexternal to the alert database may also be used to augment the machinelearning. The resulting trained model 604, 612 is exported to thesecurity threat monitoring system (operation 610).

FIG. 7 depicts how the training model is used to facilitate reducing thetime necessary for a security analyst to identify, categorize and handlethe alert. At step 700, a generated threat alert is received forhandling. As noted at 702, threat alerts are generated by the STEM, andother security systems and devices in the data center. At step 704, thesystem intakes the alert, and extracts a set of attributes to be used asinputs to the training model. At step 706, the training model is runagainst the attributes that are input; in response, a threat dispositionscore (TDS) is generated (step 708). At step 710, the system outputs aprediction, which typically comprises the TDS and information about howthe score was computed. At step 712, the alert is “enriched” with thisinformation, typically by provided the security analyst a threat details“view” (in the threat monitoring console, or otherwise). Using the alertthat has been enriched in this manner, the analyst is able to make amore information decision about the alert, and much faster. This isdepicted at step 714.

FIG. 8 depicts how the scoring model is updated as learning is on-going.In the depicted scenario, the scoring model 800 is used to generate theenriched alert 802. As a result of viewing the enriched alert, theanalyst has investigated the alert and taking an action 804, typicallyeither closing the alert 806 (as a false positive), or escalating theissue 808. (Other options depending on the SOC implementation, policiesand procedures, etc., may also be carried out). When the matter isescalated, the customer (directly or indirectly via the SOC or othersystem(s)) may take action 810, typically either to close the ticket 812(as a false positive), or by remediating the threat 814. In either case,the action 810 is reported back (step 816) to the alert database, andoptionally to a ticket database (and other monitoring and reportingsystems). At step 818, and as the alert handling continues, the learningprocess is then repeated. The result is a re-trained (or updated)scoring model 820, which is then used going forward. There-training/updating 818 may be carried out periodically, in response togiven occurrences (a threshold of false positive alerts being reached),or some combination thereof. Preferably, the re-training of the scoringmodel occurs continuously as new data points (new alerts and their alerthandling workflow) are received and stored in the alerts database.

FIG. 9 depicts how attributes for the training model are configured. Asdepicted, and as previously described, preferably the alert data(include historic data regarding alert handling) is exported from thealert database 900 and, at step 902, used by the machine learningalgorithm(s) to train the scoring model 904. The particular attributeconfiguration 906 used for this purpose may be pre-configured (e.g.,using a template, rule or policy) or user-specified (e.g., by a givensecurity analyst at some level) using a simple GUI interface provided inthe threat monitoring console. By configuring attributes in this manner,the training can be adjusted dynamically as the scoring model continuesto be re-trained/updated. In one preferred approach, the system providesthe analyst a set of drop-down configuration menus (or selections) fromwhich the user configures which attributes should be considered for thetraining data set, with the goal of improving the threat dispositionscoring accuracy.

Applying this approach here, a set of prediction models for training themachine may be generated as follows. Preferably, data set segregationinvolves developing a training set, a testing set, and selectingappropriate machine learning methods, such as random sampling withoutreplacement. In one embodiment, feature selection for the training setincludes a set of predictors (e.g., customer ID, rule names, alertcreation time, source geo, destination geo, client industry, and eventvendor), and a response variable (SOC alert status, e.g., closed orescalated). The machine learning models include, without limitation, oneor more of the following: gradient boosting models (GBM), extremegradient boosting (XGBOOST), and boosted classification trees (ADA).Representative tuning parameters include, without limitation,cross-validation using repeated sampling (e.g., using 10 repeats), slowand fast learning rates, a given number (e.g., 50, 100 and 150) trees,and a minimum number (e.g., 20) observations per node. Model performancemay be evaluated in any known manner, e.g., for accuracy, sensitivity,specificity, positive and negative prediction rates, area under areceiver operator characteristic (ROC) curve, and the like. Knownmachine learning methods may be used for this purpose.

A threat detection score (TDS) may be absolute or relative, and it maybe characterized by a number, a percentage, or the like. A particularTDS also typically has associated therewith a confidence value or level(e.g., high), as well as information detailing how the TDS was computed.For any particular threat that is the subject of an alert, anappropriate TDS is computed and output to the security analyst tofacilitate the analyst identifying, categorizing and/or prioritizing thealert for response. As a particular TDS confidence level approaches somedefined value (e.g. a configurable threshold), the system may then becontrolled automatically to implement a given remediation or mitigationoperation. Typically, alerts are classified by type, and there may asingle TDS score associated with all alerts within a type, althoughpreferably a TDS is associated to each alert including those that sharean alert type.

Typically, there is a scoring model per alert type, but this is notrequired.

FIG. 10 depicts a representative alert screen provided to an analyst. Inthis example, the TDS is displayed via two values—“recommended action”and “recommended action confidence.” These values work together tosuggest to the analyst the recommended action for the Alert underinvestigation, based on the historically-trained model, together withthe confidence of this recommendation.

The historical disposition data for an alert (or alert type) that isutilized during the machine learning may be quite varied, e.g., use casedocumentation, rule documentation, response procedure documentation,alert documentation, security incident documentation, securityintelligence feeds documentation, contextual data, mitigationdocumentation (short term fix), remediation documentation (long term fixto prevent recurrence), previous errors, quality control data, feedbackon alert handling (e.g., from other higher-level analysts), andcombinations of the above.

The approach as described above provides significant advantages.Foremost, by providing the security analyst a TDS and supportinginformation, the time required for threat investigation and validationis significant reduced, especially in cases where the TDS has a highdegree of confidence and, as a consequence, the analyst is then able tomore readily and efficiently close the alert, e.g., as a false positive.As additional alerts run through the model and the model is updated, thesystem learns continuously, all without direct human intervention or anyneed to modify static scoring algorithms. Moreover, by providing theanalyst (or some other permitted user) the ability to modify theconfiguration of the attributes of a particular threat alert, thescoring model is updated efficiently and as needed or desired, withoutany need to modify the system programmatically. As another advantage,and as noted above, as threat disposition scoring gets close to veryhigh (e.g., close to 100%) confidence, the remediation of threatresponse may then be automated, eliminating the need for any furtherinvestigation or manual response. Another advantage is the improvementin the overall accuracy and reduced error rates that improve customersatisfaction. Over a period of time, this results in productivityimprovements with reduced need to hire more analysts, even as alertvolume increases.

By implementing the approach herein, and as noted above, a securityanalyst that receives a TDS (e.g., with an appropriate confidence level)need not even perform what might be considered a routine furtherinvestigation of the alert, and instead respond as if the investigationwere already completed. Thus, by relying on the TDS, an analyst mightrespond to an alert immediately to the effect of “Escalate as a realattack” or “Close as false positive.” The scoring model learns primarilybased on attributes of the attack and, more importantly, on theknowledge (and context) available to the system based on prior similaractivity, including how “close” a prior prediction may have been or howsuccessful any final outcome might have been. Information about alertdisposition outcomes preferably are returned to the system (e.g., byhigher level analysts, by customers, or by other systems) and then usedto further refine the scoring model. Thus, the security model may beupdated to take into consideration any knowledge/context availableregarding what the “final outcome” was with respect to a particularalert as previously vetted by an analyst and/or rated by the customer.The approach herein in effect predicts that given certain types ofattacks and the related knowledge available to the system, that aparticular alert represents a high (or low) probability of being a realthreat. Because it is machine learning-based, the approach is primarilyfully automated (with the exception of attribute configuration, whichmay be manually-supported), thus obviating manual investigation of thealert details for many type(s) of alert. Essentially, the approachenables the analyst to streamline his or her analysis and even in somecases to avoid having to do any intermediate analysis, instead providingan appropriate and timely response.

Generalizing, the technique provides for a platform that uses historicalthreat remediation and customer feedback data to enrich attack detailsfor a Security Operations Center. The approach of enriching a threatalert with a machine learning-based threat disposition score (TDS) (and,optionally, associated supporting data) provides the security analystwith insight on an appropriate disposition for a received alert. Asnoted, the machine learning provides an extensive analysis of previousalerts, e.g., those that are of a similar type to the current alertbeing evaluated. By significantly reducing the analyst's time to resolvethe alert, the technique provides significant productivity and threatdisposition results over prior art techniques, such as static scoringalgorithms that do not take into account historical context, feedback onalert handling from senior analysts, and so forth. Based on the threatTDS, the analyst is able to improve the accuracy of his or her handlingon the alert, and the approach herein also reduces the amount of timeneeded to investigate an alert (e.g. when the alert is a falsepositive).

This subject matter may be implemented as-a-service. The subject mattermay be implemented within or in association with a data center thatprovides cloud-based computing, data storage or related services. Themachine learning (ML) functionality may be provided as a standalonefunction, or it may leverage functionality from other ML-based productsand services including, without limitation, a Question-Answer basedNatural Language Processing (NLP) system, products, device, program orprocess.

As noted above, the machine learning may utilize information in additionto the alert information drawn from the alert database. Thus, a machinelearning algorithm may also take advantage of consolidated security andthreat intelligence information from both structured and unstructureddata sources. Structured data sources provide security and threatintelligence information about “what/who are bad,” but typically suchdata sources lack in-depth knowledge about the threats, as well asactionable insights about how to address specific situations. Typically,structured data sources are carefully curated by domain experts.Examples include, without limitation, IBM X-Force Exchange, Virus Total,blacklists, Common Vulnerability Scoring System (CVSS) scores, andothers. Unstructured data sources, in contrast, provide much morecontextual information, such as why particular IP addresses or URLs arebad, what they do, how to protect users from known vulnerabilities, andthe like. Examples of such unstructured data sources include, withoutlimitation, threat reports from trusted sources, blogs, tweets, amongothers. Thus, the threat disposition analysis and modeling system ofthis disclosure may include a technique to consolidate security andthreat intelligence information obtained from both structured andunstructured data sources.

In a typical use case, a SIEM or other security system has associatedtherewith a interface that can be used to render the TDS visually, tosearch and retrieve relevant information from alert database, and toperform other known input and output functions with respect thereto.

As noted above, the approach herein is designed to be implemented in anautomated manner within or in association with a security system, suchas a SIEM.

The alert information itself may be processed using a question andanswer (Q&A) system, such as a natural language processing (NLP)-basedartificial intelligence (AI) learning machine. A machine of this typemay combine natural language processing, machine learning, andhypothesis generation and evaluation; it receives queries and providesdirect, confidence-based responses to those queries. A Q&A solution suchas IBM Watson may be cloud-based, with the Q&A function delivered“as-a-service” (SaaS) that receives NLP-based queries and returnsappropriate answers. In an alternative embodiment, the Q&A system may beimplemented using IBM LanguageWare, a natural language processingtechnology that allows applications to process natural language text.LanguageWare comprises a set of Java libraries that provide various NLPfunctions such as language identification, text segmentation andtokenization, normalization, entity and relationship extraction, andsemantic analysis.

The functionality described in this disclosure may be implemented inwhole or in part as a standalone approach, e.g., a software-basedfunction executed by a hardware processor, or it may be available as amanaged service (including as a web service via a SOAP/XML interface).The particular hardware and software implementation details describedherein are merely for illustrative purposes are not meant to limit thescope of the described subject matter.

More generally, computing devices within the context of the disclosedsubject matter are each a data processing system (such as shown in FIG.2 ) comprising hardware and software, and these entities communicatewith one another over a network, such as the Internet, an intranet, anextranet, a private network, or any other communications medium or link.The applications on the data processing system provide native supportfor Web and other known services and protocols including, withoutlimitation, support for HTTP, FTP, SMTP, SOAP, XML, WSDL, UDDI, andWSFL, among others. Information regarding SOAP, WSDL, UDDI and WSFL isavailable from the World Wide Web Consortium (W3C), which is responsiblefor developing and maintaining these standards; further informationregarding HTTP, FTP, SMTP and XML is available from Internet EngineeringTask Force (IETF). Familiarity with these known standards and protocolsis presumed.

The scheme described herein may be implemented in or in conjunction withvarious server-side architectures including simple n-tier architectures,web portals, federated systems, and the like. The techniques herein maybe practiced in a loosely-coupled server (including a “cloud”-based)environment.

Still more generally, the subject matter described herein can take theform of an entirely hardware embodiment, an entirely software embodimentor an embodiment containing both hardware and software elements. In apreferred embodiment, the function is implemented in software, whichincludes but is not limited to firmware, resident software, microcode,and the like. Furthermore, as noted above, the identity context-basedaccess control functionality can take the form of a computer programproduct accessible from a computer-usable or computer-readable mediumproviding program code for use by or in connection with a computer orany instruction execution system. For the purposes of this description,a computer-usable or computer readable medium can be any apparatus thatcan contain or store the program for use by or in connection with theinstruction execution system, apparatus, or device. The medium can be anelectronic, magnetic, optical, electromagnetic, infrared, or asemiconductor system (or apparatus or device). Examples of acomputer-readable medium include a semiconductor or solid state memory,magnetic tape, a removable computer diskette, a random access memory(RAM), a read-only memory (ROM), a rigid magnetic disk and an opticaldisk. Current examples of optical disks include compact disk-read onlymemory (CD-ROM), compact disk-read/write (CD-R/W) and DVD. Thecomputer-readable medium is a tangible item.

The computer program product may be a product having programinstructions (or program code) to implement one or more of the describedfunctions. Those instructions or code may be stored in a computerreadable storage medium in a data processing system after beingdownloaded over a network from a remote data processing system. Or,those instructions or code may be stored in a computer readable storagemedium in a server data processing system and adapted to be downloadedover a network to a remote data processing system for use in a computerreadable storage medium within the remote system.

In a representative embodiment, the threat disposition and modelingtechniques are implemented in a special purpose computer, preferably insoftware executed by one or more processors. The software is maintainedin one or more data stores or memories associated with the one or moreprocessors, and the software may be implemented as one or more computerprograms. Collectively, this special-purpose hardware and softwarecomprises the functionality described above.

While the above describes a particular order of operations performed bycertain embodiments of the invention, it should be understood that suchorder is exemplary, as alternative embodiments may perform theoperations in a different order, combine certain operations, overlapcertain operations, or the like. References in the specification to agiven embodiment indicate that the embodiment described may include aparticular feature, structure, or characteristic, but every embodimentmay not necessarily include the particular feature, structure, orcharacteristic.

Finally, while given components of the system have been describedseparately, one of ordinary skill will appreciate that some of thefunctions may be combined or shared in given instructions, programsequences, code portions, and the like.

The techniques herein provide for improvements to another technology ortechnical field, e.g., security incident and event management (SIEM)systems, other security systems, as well as improvements toautomation-based cybersecurity analytics.

Having described the invention, what we claim is as follows:
 1. A methodfor threat disposition analysis, comprising: responsive to receipt of asecurity threat identified in an alert, retrieving a threat dispositionscore (TDS), the threat disposition score generated from a machinelearning scoring model built from information about historical securitythreats, including historical disposition of one or more alertsassociated with the historical security threats, the TDS based in parton an effectiveness of a prior calculated TDS to predict a particularhistorical disposition associated with the alert; augmenting the alertto include the threat disposition score to generate an enriched alert;and presenting the enriched alert for further handling; wherein thehistorical disposition of at least one alert comprises feedback from asecond security analyst on handling of the at least one alert by a firstsecurity analyst.
 2. The method as described in claim 1 wherein enrichedalert also includes historical information about how the security threathas been handled previously.
 3. The method as described in claim 1wherein the feedback is generated following the at least one alerthaving been escalated from the first security analyst to the secondsecurity analyst.
 4. The method as described in claim 1 furtherincluding building the machine learning scoring model, wherein themachine learning scoring model also is built from a set of attributesregarding an alert.
 5. The method as described in claim 4 furtherincluding receiving data configuring the set of attributes.
 6. Themethod as described in claim 1 further including updating the machinelearning scoring model.
 7. The method as described in claim 1, furthercomprising: providing a confidence level associated with the TDS; andresponsive to the confidence level reaching a threshold, automaticallyperforming a set of one or more actions to respond to the securitythreat.
 8. The method as described in claim 1 wherein the furtherhandling is one of: closing the security threat as a false positive, andescalating the security threat.
 9. An apparatus, comprising: aprocessor; computer memory holding computer program instructionsexecuted by the processor for threat disposition analysis, the computerprogram instructions operative to: retrieve a threat disposition score(TDS) in response to receipt of a security threat identified in analert, the threat disposition score generated from a machine learningscoring model built from information about historical security threats,including historical disposition of one or more alerts associated withthe historical security threats, the TDS based in part on aneffectiveness of a prior calculated TDS to predict a particularhistorical disposition associated with the alert; augment the alert toinclude the threat disposition score to generate an enriched alert; andpresent the enriched alert for further handling; wherein the historicaldisposition of at least one alert comprises feedback from a secondsecurity analyst on handling of the at least one alert by a firstsecurity analyst.
 10. The apparatus as described in claim 9 whereinenriched alert also includes historical information about how thesecurity threat has been handled previously.
 11. The apparatus asdescribed in claim 9 wherein the feedback is generated following the atleast one alert having been escalated from the first security analyst tothe second security analyst.
 12. The apparatus as described in claim 9wherein the computer program instructions are further operative to buildthe machine learning scoring model, wherein the machine learning scoringmodel also is built from a set of attributes regarding an alert.
 13. Theapparatus as described in claim 12 wherein the computer programinstructions also are operative to receive data configuring the set ofattributes.
 14. The apparatus as described in claim 9 wherein thecomputer program instructions also are operative to update the machinelearning scoring model.
 15. The apparatus as described in claim 9wherein the computer program instructions also are operative to: providea confidence level associated with the TDS; and responsive to theconfidence level reaching a threshold, automatically perform a set ofone or more actions to respond to the security threat.
 16. The apparatusas described in claim 9 wherein the further handling is one of: closingthe security threat as a false positive, and escalating the securitythreat.
 17. A computer program product in a non-transitory computerreadable medium for use in a data processing system for threatdisposition analysis, the computer program product holding computerprogram instructions that, when executed by the data processing system,are operative to: retrieve a threat disposition score (TDS) in responseto receipt of a security threat identified in an alert, the threatdisposition score generated from a machine learning scoring model builtfrom information about historical security threats, including historicaldisposition of one or more alerts associated with the historicalsecurity threats, the TDS based in part on an effectiveness of a priorcalculated TDS to predict a particular historical disposition associatedwith the alert; augment the alert to include the threat dispositionscore to generate an enriched alert; and present the enriched alert forfurther handling; wherein the historical disposition of at least onealert comprises feedback from a second security analyst on handling ofthe at least one alert by a first security analyst.
 18. The computerprogram product as described in claim 17 wherein enriched alert alsoincludes historical information about how the security threat has beenhandled previously.
 19. The computer program product as described inclaim 17 wherein the feedback is generated following the at least onealert having been escalated from the first security analyst to thesecond security analyst.
 20. The computer program product as describedin claim 17 wherein the computer program instructions are furtheroperative to build the machine learning scoring model, wherein themachine learning scoring model also is built from a set of attributesregarding an alert.
 21. The computer program product as described inclaim 20 wherein the computer program instructions also are operative toreceive data configuring the set of attributes.
 22. The computer programproduct as described in claim 17 wherein the computer programinstructions also are operative to update the machine learning scoringmodel.
 23. The computer program product as described in claim 17 whereinthe computer program instructions also are operative to: provide aconfidence level associated with the TDS; and responsive to theconfidence level reaching a threshold, automatically perform a set ofone or more actions to respond to the security threat.
 24. The computerprogram product as described in claim 17 wherein the further handling isone of: closing the security threat as a false positive, and escalatingthe security threat.
 25. A security threat analysis platform,comprising: one or more hardware processors; a data store holding aknowledge base of alert data, and historical alert disposition handlinginformation; and computer memory storing computer program instructionsconfigured to; compute a scoring model by applying machine learning toinformation derived from the knowledge base, the information includinghistorical security threats, including historical disposition of one ormore alerts associated with the historical security threats; respond toreceipt of a new security threat, using the scoring model to generate analert having an associated threat disposition score and confidencelevel, the threat disposition score based in part on an effectiveness ofa prior calculated threat disposition score to predict a particularhistorical disposition associated with the alert; receive and respond tohandling information for the alert; wherein the historical dispositionof the alert comprises feedback from a second security analyst onhandling of the alert by a first security analyst.
 26. The securitythreat analysis platform as described in claim 25, the alert having beenescalated to the second security analyst from the first securityanalyst, and wherein the computer program instructions configured toreceive and respond to handling information for the alert comprisecomputer program instructions further configured to: present the alertto the first security analyst responsible for addressing the newsecurity threat; receive data indicating a response by the firstsecurity analyst to the alert, the response based at least in part oninclusion of the threat disposition score; and update the scoring modelbased at least in part on the response by the first security analyst tothe alert; wherein inclusion of the threat disposition score reduces analert disposition error rate associated with the first security analyst.